Selecting Dataset

The Census Bureau only released APIs for the following time frames:

get_acs()

• American Community Survey 1-Year Data (2011-2017)
• American Community Survey 3 Year Data (2012-2013)
• American Community Survey 5-Year Data (2009-2017)

get_decennial()

• Decennial US Census (1990, 2000, 2010)

Selecting Geograpghic Scale

Geography	Definition
us	ACS only. United States
region	ACS only. Census regions are groupings of states and the District of Columbia that subdivide the United States for the presentation of census data. The Census Bureau defines four census regions and identifies each one with a single-digit census code- Northeast (1), Midwest (2), South (3), and West (4). Puerto Rico and the Island Areas are not part of any census region.
division	ACS only. Census divisions are groupings of states and the District of Columbia that subdivide the census regions for the presentation of census data. The Census Bureau defines nine census divisions and identifies each one with a single-digit census code- New England (1), Middle Atlantic (2), East North Central (3), West North Central (4), South Atlantic (5), East South Central (6), West South Central (7), Mountain (8), and Pacific (9). Puerto Rico and the Island Areas are not part of any census division.
state	States and equivalent entities are the primary governmental divisions of the United States. In addition to the 50 states, the Census Bureau treats the District of Columbia, Puerto Rico, American Samoa, the Commonwealth of the Northern Mariana Islands, Guam, and the U.S. Virgin Islands as the statistical equivalents of states for the purpose of data presentation.
county	Counties and their equivalents are the primary division of states providing complete coverage of a state. In most states, counties are the primary legal division. Louisiana has parishes; Alaska has boroughs, city and boroughs, municipalities, and census subareas; Maryland, Missouri, Nevada, and Virginia have independent cities in addition to counties; Puerto Rico has municipios; American Samoa has districts and island; the Commonwealth of the Northern Mariana Islands have municipalities; the U.S. Virgin Islands have islands; and the District of Columbia and Guam are also county equivalents.
county subdivision	County subdivisions are the primary divisions of counties and statistical equivalent entities. They are either legal entities (minor civil divisions) or statistical entities (census county divisions, census subareas, and unorganized territories). The two main types of county subdivisions are minor civil divisions (MCD) and census county divisions (CCD). MCDs are legal entities that provide governmental and/or administrative services, and are most commonly known as towns and townships. CCDs are statistical entities established cooperatively by the Census Bureau and state and local officials. Census subareas occur only in Alaska and unorganized territories (UT) are defined by the Census Bureau for areas where portions of counties, or equivalent entities, are not already included in an MCD.
tract (must include state)	Census tracts are small, relatively permanent statistical subdivisions of a county or county equivalent and generally have a population size between 1,200 and 8,000 people, with an optimum size of 4,000 people. The Census Bureau created census tracts to provide a stable set of boundaries for statistical comparison from census to census. Census tracts occasionally split due to population growth or merge when there is substantial population decline. Local governments have the opportunity to review census tract boundaries prior to each decennial census through the Participants Statistical Areas Program. In general, census tracts can either merge or split based on population changes but cannot be completely changed in order to maintain data comparability over time. If a local participant is not available, the Census Bureau will update the boundaries.
block group (must include state)	Block groups are statistical divisions of census tracts and generally contain between 600 and 3,000 people. Local participants delineate most block groups prior to each decennial census through the Participant Statistical Areas Program. If a local participant is not available, the Census Bureau will define them. Since block groups consist of a cluster of census blocks, they control block numbering. At the time of block delineation, all of the blocks that fall within a block group will start with the same number. For example, census blocks 2001, 2002, 2003, … 2999 in census tract 1310.02 belong to block group 2.
block (must include state and county)	Decennial only. Census Blocks are the smallest geographic areas that the Census Bureau uses to tabulate decennial data. Blocks are statistical areas bounded by visible features, such as streets, roads, streams, and railroad tracks, and by nonvisible boundaries, such as selected property lines and city, township, school district, and county limits. Generally, census blocks are small in area; for example, a block in a city bounded on all sides by streets. Census blocks cover the entire territory of the United States, Puerto Rico, and the Island Areas. Census blocks nest within all other tabulated census geographic entities and are the basis for all tabulated data. Blocks are defined once a decade and data are available only from the decennial census 100% data (age, sex, race, Hispanic/Latino origin, relationship to householder, and own/rent house). In between decennial censuses, blocks may split due to boundary changes but these blocks do not have any statistical data associated with them available.
place	The Census Bureau identifies two types of places; incorporated places and census designated places (CDP). Incorporated places are legal areas and provide governmental functions for a concentration of people. They are usually cities, towns, villages, or boroughs. CDPs are statistical areas delineated to provide data for a settled community, or concentration of population, identifiable by name but not legally incorporated. There is no minimum population threshold for a CDP and they are delineated with the help of local governments, through the Participant Statistical Areas Program, usually once a decade.
metropolitan statistical area/ micropolitan statistical area	The Office of Management and Budget (OMB) defines metropolitan and micropolitan statistical areas, collectively known as core based statistical areas (CBSA), based on data from the decennial census and commuting data from the American Community Survey. Each metropolitan or micropolitan area consists of one or more whole counties and includes the counties containing the core urban area, as well as any adjacent counties that have a high degree of social and economic integration (as measured by commuting to work) with the urban core. Metropolitan statistical areas are based on urbanized areas of 50,000 or more people and micropolitan statistical areas are based on urban clusters of at least 10,000 but less than 50,000 people.
combined statistical area	ACS only. Combined statistical areas (CSA) consist of two or more adjacent metropolitan and micropolitan statistical areas that have substantial employment interchange. The metropolitan and micropolitan statistical areas that combine to create a CSA retain separate identities within the larger CSA.
urban area	ACS only. Urban Area is the term for urbanized areas (UAs) and urban clusters (UCs). UAs consist of densely developed area that contains 50,000 or more people. UCs consist of densely developed area that has a least 2,500 people but fewer than 50,000 people. The Census Bureau defines urban areas once a decade after the population totals for the decennial census are available, and classifies all territory, population, and housing units located within a UA or UC as urban and all area outside of a UA or UC as rural. Urban areas are used as the cores on which core based statistical areas are defined.
congressional district	Congressional districts are electoral districts that elect a single member of congress to the House of Representatives. There are 435 congressional districts in the U.S. and the Census Bureau’s decennial census counts determine the number of congressional districts given to each state. The District of Columbia, Puerto Rico, and each Island Area (American Samoa, Guam, Northern Mariana Islands, and U.S. Virgin Islands) are assigned one nonvoting delegate each.
public use microdata area	ACS only. Public use microdata areas (PUMAs) are geographic areas defined to be used with public use microdata sample (PUMS) files. PUMAs are a collection of counties or tracts within counties with more than 100,000 people, based on the decennial census population counts. State partners define PUMAs once a decade after the decennial census. Data for PUMAs are available from the American Community Survey (ACS). PUMS files are available from the ACS and decennial census.
zip code tabulation area	ZIP Code® tabulation areas (ZCTAs) are generalized areal representations of U.S. Postal Service (USPS) ZIP Code service areas. The Census Bureau collects ZIP Code data for housing units and many non-residential addresses from the USPS and from various field operations. Based on this ZIP Code data, the Census Bureau aggregates census blocks with the same ZIP Code to form ZCTAs. The Census Bureau then labels each ZCTA with the five-digit ZIP Code number used by the USPS.

Selecting Variables

# Pull all variables
acs_variables <- load_variables(2016, "acs5")

head(acs_variables)

# In R Studio, use view and filter to search for a variable
View(acs_variables)

# Search for variables in a known table
acs_variables %>%
  filter(str_detect(name,"B19001")) %>%
  head()

# Search for a known variable using a keyword
acs_variables %>%
  filter(str_detect(label,fixed("race", ignore_case = TRUE))) %>%
  head()

vars1 <- acs_vars("B17010B_{c(4, 11, 17, 24, 31, 37)*}")

vars2 <- acs_vars("B01001_{c(3:6, 27:30)*}")

Getting Data

# Decennial 
get_decennial(geography = "state", 
              variables = c(Population = "P001001"),
              year = 2010,
              output = "wide") %>%
  head(5)

# American Community Survey
get_acs(geography = "tract",
        state = "NY", 
        variables = c(Female_Newborn = "B01001_027"),
        year = 2016,
        survey = "acs5",  # acs5 = five-year estimates, acs1= one-year estimates
        output = "wide") %>%
  rename(Female_Newborn_Estimate = Female_NewbornE,
         Female_Newborn_MOE = Female_NewbornM) %>%
  head(5)

# Getting the whole table 
get_acs(geography = "county",
        state = "NY",
        table = "B19001",
        year = 2016,
        survey = "acs5",
        output = "wide") %>%
  head(3)

# Over multiple states
get_acs(geography = "tract",
        state = c("NY", "CT", "NJ", "PA", "MA", "RI"),
        variables = vars2,
        year = 2016,
        survey = "acs5",
        output = "wide") %>%
  head(3)

# Over multiple years 
map_dfr(2012:2016, function(x) {
  get_acs(geography = "state",
          state = "NY",
          variables = "B03002_001",
          year = x,
          survey = "acs5") %>% 
    mutate(year = x)})

Wrangling Data

get_acs(geography = "tract",
        state = "NY", 
        variables = vars2,
        year = 2016,
        survey = "acs5",
        output = "wide") %>%
  mutate(pop_u18_total = acs_sum("B01001_{c(3:6, 27:30)*}E"),
         pop_u18_male = acs_sum("B01001_{c(3:6)*}E"),
         pop_u18_female = acs_sum("B01001_{c(27:30)*}E")) %>%
  select(GEOID, pop_u18_total, pop_u18_male, pop_u18_female) %>%
  head(3)

## Getting data from the 2012-2016 5-year ACS

race_vars <- c(White = "B03002_003", Black = "B03002_004", Native = "B03002_005", 
               Asian = "B03002_006", HIPI = "B03002_007", Hispanic = "B03002_012")

total_pop <- "B03002_001"

ny_acs_income_data <- get_acs(geography = "county",
                              state = "NY",
                              variables = race_vars,
                              summary_var = total_pop,
                              year = 2016,
                              survey = "acs5")

ny_acs_income_data %>%
  mutate(pct = 100 * (estimate / summary_est)) %>% 
  head(3)

ny_acs_income_data %>%
  group_by(GEOID) %>%
  filter(estimate == max(estimate)) %>% 
  head()

ny_acs_income_data %>%
  group_by(GEOID) %>%
  filter(estimate == max(estimate)) %>% 
  group_by(variable) %>%
  tally()

ny_acs_income_data %>%
  filter(variable != "B19001_001") %>%  #"B19001_001" is the total count
  mutate(incgroup = case_when(
    variable < "B19001_008" ~ "below35k", 
    variable < "B19001_013" ~ "35kto75k", 
    TRUE ~ "above75k")) %>%
  group_by(NAME, incgroup) %>%
  summarize(group_est = sum(estimate)) %>%
  head(3)

ACS Margins of Error

Whereas the Decennial census is a literal census (or as close to one as possible), the ACS is based on a sampling. As such, the numbers are really estimates which also have margins of error because they are not literal counts.

ny_elder_poverty <- get_acs(geography = "tract",
                            state = "NY",
                            variables = c(male = "B17001_016", female = "B17001_030"),
                            year = 2016,
                            survey = "acs5")

moe_check <- ny_elder_poverty %>%
  filter(moe > estimate) 

nrow(moe_check) / nrow(ny_elder_poverty)

## [1] 0.8294022

# Adding variables
moe_sum(moe = c(55, 33, 44, 12, 4),
        estimate = c(1000, 125, 300, 54, 12))

## [1] 78.80355

# Multiplying variables
moe_product(est1 = 55,
            est2 = 33,
            moe1 = 12,
            moe2 = 9)

## [1] 633.9093

# Finding the ratio between variables
moe_ratio(num = 1000,
          denom = 950,
          moe_num = 200,
          moe_denom = 177)

## [1] 0.287724

# Finding the proportion of one variable by another 
moe_prop(num = 374,
         denom = 1200,
         moe_num = 122,
         moe_denom = 333)

## [1] 0.05344178

# Example 
ny_elder_poverty <- ny_elder_poverty %>%
  group_by(GEOID) %>%
  summarize(estmf = sum(estimate),
            moemf = moe_sum(moe = moe, estimate = estimate))

ny_elder_poverty %>% head(3)

moe_check <- ny_elder_poverty %>%
  filter(moemf > estmf)

nrow(moe_check) / nrow(ny_elder_poverty)

## [1] 0.6520943

Comparing Years

The Census Bureau divides the country into tracts of around 4,000 people that are “designed to be relatively homogeneous units with respect to population characteristics, economic status, and living conditions”. Tracts were introduced in 1910, but only a few cities (New York being one of them) were tracted; it wasn’t until 1990 that the whole country was tracted! Because they are based on population size, tract boundaries can change over time from Census to Census. This makes comparing indicators across time difficult since the same tract number in one dataset may refer to a different geography in a different dataset. Adjusting the tracts enables comparisons of the same area over time.

The most recent change to Census tract boundaries followed the 2010 Census. Thus, Census data (or other data reported at the Census tract level) from 1970-2000 needs to be translated into 2010 Census tracts (or 2010 and later data can be translated back to a previous year’s geography) before a comparison can be made.

The Longitudinal Tract Database (LTDB) is an open-source crosswalk that helps bridge data for Census tracts across time. Each row in the crosswalk contains a tract number from a previous year, a tract number of the current year that overlaps with the previous boundary, and the proportion of the previous tract that is within the current tract to serve as a weight. Old tracts that were split into multiple new tracts have a row for each new tract, so the column containing the old tract numbers does not contain unique values. Similarly, new tracts that are made up of several old tracts have a row for each of the old tracts that were merged, so the column containing new tract numbers does not contain unique values. Only the combination of old and new tract number columns is unique.

from_year = 2010
to_year = 2000

from_yr = str_sub(from_year,-2,-1)
to_yr = str_sub(to_year,-2,-1)

xwalk <- read_csv("path to downloaded LTDB crosswalk")

old_tracts <- get_acs(geography = "tract",
                      state = "NY", 
                      variables = c(Female_Newborn = "B01001_027"),
                      year = 2009,
                      survey = "acs5",  
                      output = "wide")

new_tracts <- old_tracts %>%
  inner_join(xwalk, by = glue("trtid{from_yr}"))%>%
  mutate(Female_Newborn = Female_Newborn * weight) %>%
  group_by(!!as.name(glue("trtid{to_yr}")))%>%
  summarise(pop_num_00 = sum(pop_num))%>%
  ungroup()

Getting Shapefiles

tigris_cache_dir("Your preferred cache directory path would go here")
options(tigris_use_cache = TRUE)

# Counties
ny_counties <- counties(state = "NY")

plot(ny_counties$geometry)

# Clipped Boundaries
ny_counties_clipped <- counties(state = "NY", cb = TRUE)

plot(ny_counties_clipped$geometry)

# Census Tracts
manhattan_tracts <- tracts(state = "NY", county = "New York", cb = TRUE)

plot(manhattan_tracts$geometry)

# Water Features 
ithica_water <- area_water(state = "NY", county = "Tompkins")

plot(ithica_water$geometry)

# Roads
ny_roads <- primary_secondary_roads(state = "NY")

plot(ny_roads$geometry)

# Census Tract Year
orange_tracts_90 <- tracts(state = "NY", county = "Orange", cb = TRUE, year = 1990)
orange_tracts_16 <- tracts(state = "NY", county = "Orange", cb = TRUE, year = 2016)

plot(orange_tracts_90$geometry)

plot(orange_tracts_16$geometry)

Plotting

plot(shapefile)

mi_tiger <- counties("MI")
mi_cb <- counties("MI", cb = TRUE)

plot(mi_tiger$geometry)
plot(mi_cb$geometry, add = TRUE, border = "red")

Joining Data*

ny_tracts <- tracts(state="NY", cb = TRUE)
nj_tracts <- tracts(state="NJ", cb = TRUE)

ny_nj_tracts <- rbind_tigris(ny_tracts, nj_tracts)

new_england <- c("ME", "NH", "VT", "MA")

ne_tracts <- map(new_england, function(x) {
  tracts(state = x, cb = TRUE)}) %>%
  rbind_tigris()

ny_counties_pop = get_acs(geography = "county",
                       state = "NY",
                       variables = "B03002_001",
                       year = 2016,
                       survey = "acs5",
                       output = "wide")

ny_counties_shp <- counties(state = "NY", cb = TRUE)

ny_counties <- left_join(ny_counties_shp, ny_counties_pop, by = "GEOID")

get_acs(geography = "tract",
        state = "NY",
        county = "New York", 
        variables = "B25077_001", 
        geometry = TRUE) #####################